Tracked object is a moving object which is captured to appear continuously on a screen and is assumed to be paid attention by a cameraman. Detection of the tracked objects, therefore, is required for generation of summarized video or extraction of key frames which are used for recognizing important objects in video.
One example of a tracked object determination device is recited in Patent Literature 1 (Japanese Patent Laying-Open No. 08-191411).
The method recited in Patent Literature 1 is to calculate a possibility of existence of a tracked object in a video segment where shooting is executed by a camera moving in a fixed direction based on a distribution of moving regions which are image regions having a vector different from a motion vector generated by motion of the camera. In this method, “a set of moving regions constantly existing in the lump” is determined to be a tracked object by using the degree of lumping of moving regions obtained by the number of pixels in a moving region, the degree of concentration, a position of the center of gravity and the degree of dispersion, or the stationary degree obtained by a rate of the number of frames including moving regions in the video segment.
Structure for determining a tracked object includes, as shown in FIG. 23, a motion vector detection unit 300 which detects a motion vector with respect to each frame of video, a panning section detection unit 301 which detects a start point and an end point of a panning section based on a motion vector of each frame of video, a moving region information detection unit 302 which detects, as a moving region, a region whose correlation degree is low between a motion compensation predicted image as a predicted image which is shifted in parallel from a past video frame by the amount of a motion vector and a current frame to extract distribution information of the region, and a scene determination unit 303 which determines, based on distribution information of a moving region detected in each panning section, that the section corresponds to a scene which tracks the object.
Patent Literature 1: Japanese Patent Laying-Open No. H08-191411.
Non-Patent Literature 1: Yousuke Torii, Seiichi Konya and Masashi Morimoto, “Extracting follow and close-up shots from moving images”, MIRU2005, pp. 24-31.
Non-Patent Literature 2: Yoshio Iwai, Shihong Lao, Osamu Yamaguchi, Takatsugu Hirayama, “A Survey on Face Detection and Face Reconition”, IPSJ SIG Technical Reports (CVIM-149), 2005, pp. 343-368.
The first problem of the related art method is that a moving object is not determined as a tracked object when a camera cannot be moved approximately with the same speed as the moving object.
Moving object cannot be determined to be a tracked object in a case, for example, where shot ends before a shifting rate of a camera comes to be equal to a shifting rate of a moving object because of shortness of the shot, where a cameraman is not allowed to predict a shifting destination of a moving object because the moving object moves at random or where a camera shifting rate varies as a moving object moves such as a case of shooting by a telephoto camera.
The reason is that whether a moving object is an object to be tracked or not is determined by a state of distribution of moving regions as an image region having a motion vector different from a motion vector generated by movement of the camera. Accordingly, when a distribution of moving regions fails to satisfy the property that “constantly exists in the lump” because the movement of the camera differs from a shifting rate of the moving object, the moving object as a set of moving regions cannot be determined to be a tracked object.
Second problem of the related art method is that a moving object cannot be judged as a tracked object before the end point of a video segment captured with a fixed direction.
The reason is that because whether a moving object is an object to be tracked or not is determined based on a rate of the number of frames in which the moving object can be stably tracked within the video segment, the moving object cannot be determined to be a tracked object unless a length of the video segment is obtained.